Automatic Indexing by Discipline and High-Level Categories: Methodology and Potential Applications

This paper first describes the methodology of journal descriptor (JD) indexing, based on human indexing at the journal level using only 127 descriptors, and applying statistical methods that associate this journal indexing with text words in a training set of MEDLINE® citations. These associations form the basis for automatic indexing of documents outside the training set. The paper then presents the new technique of semantic type (ST) indexing, based on JD indexing associated with each of 134 ST’s, and applying the standard cosine coefficient measure to compare the similarity between the JD indexing of a document and the JD indexing of each ST. The ST indexing of the document is the list of ST’s ranked in decreasing order of similarity between the JD indexing of the document and the JD indexing of the ST’s. Discussion of the potential usefulness and application of the very general indexing provided by JD’s and ST’s comprises the remainder of the paper. JD’s have been used for more than thirty years to search MEDLINE by discipline, and discipline-based indexing is in evidence on the Web. It is suggested, with several examples, that ST’s may convey a unique slant of a document’s content not normally represented in standard indexing vocabularies. Use of ST indexing to rank retrieved output is mentioned as a possible application. Notwithstanding the importance of methodology and performance issues, the intent of this paper is to explore questions of the potential utility and applicability of JD and ST indexing.